Human type 5 17beta-hydroxysteroid dehydrogenase plays a crucial role in local androgen formation in prostate tissue. Several chemicals were synthesized and tested for their ability to inhibit this enzyme, and a series of estradiol derivatives bearing a lactone on the D-ring were found to inhibit its activity efficiently. The crystal structure of the type 5 enzyme in complex with NADP and such a novel inhibitor, EM1404, was determined to a resolution of 1.30 A. Significantly more hydrogen bonding and hydrophobic interactions were defined between EM1404 and the enzyme than in the substrate ternary complex. The lactone ring of EM1404 accounts for important interactions with the enzyme, whereas the amide group at the opposite end of the inhibitor contributes to the stability of three protein loops involved in the construction of the substrate binding site. EM1404 has a strong competitive inhibition, with a Ki of 6.9+/-1.4 nM, demonstrating 40 times higher affinity than that of the best inhibitor previously reported. This is observed despite the fact that the inhibitor occupies only part of the binding cavity. Attempts to soak the inhibitor into crystals of the binary complex with NADP were unsuccessful, yielding a structure with a polyethylene glycol fragment occupying the substrate binding site. The relative crystal packing is discussed. Combined studies of small molecule inhibitor synthesis, x-ray crystallography, enzyme inhibition, and molecular modeling make it possible to analyze the plasticity of the substrate binding site of the enzyme, which is essential for developing more potent and specific inhibitors for hormone-dependent cancer therapy.